The present invention relates to improvements in artificial systems for vision and the like which produce sensations such as phosphenes in the subjects. (The term xe2x80x9csubjectxe2x80x9d is here used to refer to an individual, such as a totally or partially blind individual, using the system in question.)
It has long been known that a blind individual can be made to perceive a truly visual sensation when stimulations such as brief pulses are applied to electrodes implanted in contact with the nervous system of the subject. A pulse or train of pulses directed to a given electrode connected to a unique location results in the stimulated perception by the subject of a spot or cluster of light, called a phosphene, at its own particular location. A relatively small number of phosphenes created by stimulation of electrodes appropriately selected from an implanted array will present to the subject a pattern of light corresponding to that which, at any given instant, a camera aimed by the subject, as by being attached to his head, xe2x80x9cseesxe2x80x9d. (The term xe2x80x9ccameraxe2x80x9d is here used in its broadest sense to denote a device which senses [xe2x80x9cseesxe2x80x9d] a particular object or series of objects and produces a corresponding set of signals.)
Those signals are, in a known artificial vision system, conveyed through the subject""s skull to a series of individual electrodes in contact with predetermined spots at appropriate locations of the subject""s brain cortex, whether the visual cortex or the association cortex, each such electrode, when appropriately energized by a received signal, producing a uniquely located phosphene sensed by the subject. The selection of individual electrodes to be signal-energized at any particular moment defines the stimulated image corresponding to what the camera senses. The subject, aware of the phosphenes existing at any given moment, thus xe2x80x9cseesxe2x80x9d something which to him represents a particular object or shape.
Recently significant improvements have been made in such systems, especially but not exclusively artificial vision systems, giving to the subject an increasingly effective xe2x80x9csightxe2x80x9d, and providing to the supervisor of such systems and the designer of improvements therein a greater facility in understanding what the subject actually xe2x80x9cseesxe2x80x9d and to what extent that corresponds to actuality, so that potential improvements can more expeditiously be carried out and evaluated.
To create in the subject""s brain a set of sensations which represent a particular object or shape has presented a problem. To do so with equipment which can be conveniently carried about by the subject greatly complicates the problem. This patent relates to recent improvements in systems of this type which materially expand their usability and practicability. Those improvements fall into two closely related categoriesxe2x80x94matters directly affecting what the subject xe2x80x9cseesxe2x80x9d, and matters improving the ability of the designer-supervisor of the system to ascertain and evaluate how the system is actually functioning so as to guide him in making further improvements.
To these ends it has been found that the intelligibility and meaningfulness to the subject of the phosphene-produced image is significantly enhanced when (a) that image is caused to be a negative rather than a positive of what the camera sensesxe2x80x94in other words, the dark and light portions of the camera-sensed object are reversed into light and dark signal portions respectivelyxe2x80x94and (b) the edges of the viewed object are brightly outlined.
A meaningful image can be produced in the nervous system of a subject using only a limited number of available electrodes, thus reducing the signal-producing and -manipulating requirement of the system. Moreover, it has been found that the effectiveness of such an image can advantageously be improved by applying, for a given camera signal, a series of similar pulses to each operative electrode. Through the use of a multiplexing circuit a given pulse is applied sequentially to each of a series of selected electrodes, that pulse is then reapplied sequentially to that series of electrodes, and so on, so that a given camera signal is effectively utilized to energize a group of selected electrodes with sequential pulses.
Primarily because of weight and size considerations, the camera used in a system of the type under discussion must be extremely simple. For example, in the artificial vision system presently in use the camera is a miniaturized TV camera carried by, and contained within, a single lens area of what appears to be an ordinary pair of sunglasses. Optically such a camera is not very versatile and in particular cannot optically magnify or modify that which it senses. However, I have found that if approximate circuitry is provided between the camera and the electrodes on the subject""s nervous system which will controllably magnify the amplitude of the signal produced thereby, that will in effect magnify the perceived object and thus produce a xe2x80x9czoomxe2x80x9d effect upon the phosphene image, the area of which is fixed. That amplification variation can be under the control of the subject, who can then produce the xe2x80x9czoomxe2x80x9d effect whenever and to whatever extent desired.
One problem with phosphene-produced images is that they appear to be at no particular distance from the subject but instead to more or less float in space, whether the object they represent is close to or at a distance from the subject. This clearly limits the effectiveness of the image in advising the subject accurately with respect to the object viewed. Rangefinders with variable audible output are known and could be used by blind subjects but they have the disadvantage of interfering with the subject""s normal hearing or other senses. That disadvantage is avoided, in accordance with the present invention, by causing the distance sensed by the rangefinder to produce in the nervous system a visible distance indicationxe2x80x94illumination of specific phosphenes to represent specific distances (e.g., near, medium or far) or periodic variations in the produced stimulation, for example, a variation in intensity in visible stimulation, and preferably a blinking on and off, at a rate corresponding to the sensed distance, thereby conveying distance-intelligence to the subject while at the same time not significantly interfering with the visual representation then being conveyed to him of the object being viewed nor with his normal auditory activities.
In particular the data processing carried out by the system in question takes the signal produced by the camera, feeds it through a link to a sub-notebook computer, obtains a corresponding output from the sub-notebook computer and feeds that output to a micro-controller, the corresponding output of the micro-controller being amplified before being applied selectively to the electrodes in the subject""s nervous system.
There are several ways in which the supervisor-designer of the system may be kept aware in detail of the manner in which that system is operating in order for that individual to be able to conceive of and implement improvements. To that end, in the system of the present invention the camera is provided with means for indicating to the supervisor-designer where the camera is pointed at any given point in time. That pointer may conveniently comprise a small laser source attached to the subject""s sunglasses which produces a visible narrow light beam which will impinge upon the object being viewed by the subject. Further to the same end, the supervisor-designer is provided with a dual display system which simultaneously exhibits for purposes of comparison what the camera actually sees and the corresponding configuration of the stimulations produced in the subject""s brain.
Essential to such an evaluation in a particular artificial vision system is an accurate map setting forth, as precisely as possible, the location of each phosphene or group of phosphenes in the subject""s xe2x80x9csightxe2x80x9d corresponding to a given electrode. Phosphene maps have been made in the past by energizing a particular electrode and asking the subject to state or indicate where the phosphene thus produced is located. This is not as easy as it sounds because phosphenes as a group sometimes move about in the subject""s visual field and because objective statements from the subject as to where a particular phosphene is located are often vague and sometimes misleading. According to the present invention a more accurate phosphene map, not as subject to such vagaries, is obtained by first energizing two selected electrodes to produce two separated reference phosphenes to define a reference line, such as a vertical line, then individually energizing additional electrodes each producing its own associated phosphene, and obtaining from the subject an estimate of the position of that additional phosphene relative to the two originally selected referenced phosphenes and to the reference line which the latter define.
Each of these improvements enables and enhances the functioning of a system such as an artificial vision system effective to promote individual mobility. The cumulative effect of these improvements gives rise to a significant step forward in such artificial sensing systems.